Score-Based Models — Working Notes

we dont actually need \( p_{data}(x) \) to sample from the model. We just need a transformation: to transform images to data space.

Just to maximize the log likelihood in the data space. We only need \( \nabla_{x}\log p_{data}(x) \), this is called score function, to make the probability of some sampled image maximized, through SGD.

Target: \( s_{\theta}(x) \approx \nabla_{x}\log p_{data}(x) \)

minimize an L2:

\[ \mathbb{E}_{x\sim p_{data}(x)}[ \| \nabla_{x} \log p_{data}(x) - s_{\theta}(x) \| ] \]

How to train? How to get ground truth score \(\nabla\)?